26 research outputs found
NerfAcc: Efficient Sampling Accelerates NeRFs
Optimizing and rendering Neural Radiance Fields is computationally expensive
due to the vast number of samples required by volume rendering. Recent works
have included alternative sampling approaches to help accelerate their methods,
however, they are often not the focus of the work. In this paper, we
investigate and compare multiple sampling approaches and demonstrate that
improved sampling is generally applicable across NeRF variants under an unified
concept of transmittance estimator. To facilitate future experiments, we
develop NerfAcc, a Python toolbox that provides flexible APIs for incorporating
advanced sampling methods into NeRF related methods. We demonstrate its
flexibility by showing that it can reduce the training time of several recent
NeRF methods by 1.5x to 20x with minimal modifications to the existing
codebase. Additionally, highly customized NeRFs, such as Instant-NGP, can be
implemented in native PyTorch using NerfAcc.Comment: Website: https://www.nerfacc.co
Nerfbusters: Removing Ghostly Artifacts from Casually Captured NeRFs
Casually captured Neural Radiance Fields (NeRFs) suffer from artifacts such
as floaters or flawed geometry when rendered outside the camera trajectory.
Existing evaluation protocols often do not capture these effects, since they
usually only assess image quality at every 8th frame of the training capture.
To push forward progress in novel-view synthesis, we propose a new dataset and
evaluation procedure, where two camera trajectories are recorded of the scene:
one used for training, and the other for evaluation. In this more challenging
in-the-wild setting, we find that existing hand-crafted regularizers do not
remove floaters nor improve scene geometry. Thus, we propose a 3D
diffusion-based method that leverages local 3D priors and a novel density-based
score distillation sampling loss to discourage artifacts during NeRF
optimization. We show that this data-driven prior removes floaters and improves
scene geometry for casual captures.Comment: ICCV 2023, project page: https://ethanweber.me/nerfbuster
Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions
We propose a method for editing NeRF scenes with text-instructions. Given a
NeRF of a scene and the collection of images used to reconstruct it, our method
uses an image-conditioned diffusion model (InstructPix2Pix) to iteratively edit
the input images while optimizing the underlying scene, resulting in an
optimized 3D scene that respects the edit instruction. We demonstrate that our
proposed method is able to edit large-scale, real-world scenes, and is able to
accomplish more realistic, targeted edits than prior work.Comment: Project website: https://instruct-nerf2nerf.github.io; v1. Revisions
to related work and discussio